Blog

Threat Finds

How AI Caught APT41 Exploiting Vulnerabilities

Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image
01
Apr 2020
01
Apr 2020
Analyzing how the cyber-criminal group APT41 exploited a zero-day vulnerability, we show how Darktrace’s AI detected and investigated the threat immediately.

Executive summary

  • Darktrace detected several highly targeted attacks in early March, well before any associated signatures had become available. Two weeks later, the attacks were attributed to Chinese threat-actor APT41.
  • APT41 exploited the Zoho ManageEngine zero-day vulnerability CVE-2020-10189. Darktrace automatically detected and reported on the attack in its earliest stages, enabling customers to contain the threat before it could make an impact.
  • The intrusions described here were part of a wider campaign aiming to gain initial access to as many companies as possible during the window of opportunity presented by CVE-2020-10189.
  • The reports generated by Darktrace highlighted and delineated every aspect of the incident in the form of a meaningful security narrative. Even a junior responder could have reviewed this output and acted on this zero-day APT attack in under 5 minutes.

Fighting APT41’s global attack

In early March, Darktrace detected several advanced attacks targeting customers in the US and Europe. A majority of these customers are in the legal sector. The attacks shared the same Techniques, Tools & Procedures (TTPs), targeting public-facing servers and exploiting recent high-impact vulnerabilities. Last week, FireEye attributed this suspicious activity to the Chinese cyber espionage group APT41.

This campaign used the Zoho ManageEngine zero-day vulnerability CVE-2020-10189 to get access to various companies, but little to no follow-up was detected after initial intrusion. This activity indicates a broad-brush campaign to get initial access to as many target companies as possible during the zero-day window of opportunity.

The malicious activity observed by Darktrace took place late on Sunday March 8, 2020 and in the morning of March 9, 2020 (UTC), broadly in line with office hours previously attributed to the Chinese cyber espionage group APT41.

The graphic below shows an exemplary timeline from one of the customers targeted by APT41. The attacks observed in other customer environments are identical.

Timeline of the APT41 attack
Figure 1: A timeline of the attack

Technical analysis

The attack described here centered around the Zoho ManageEngine zero-day vulnerability CVE-2020-10189. Most of the attack appears to have been automated.

We observed the initial intrusion, several follow-up payload downloads, and command and control (C2) traffic. In all cases, the activity was contained before any later steps in the attack lifecycle, such as lateral movement or data exfiltration, were identified.

The below screenshot shows an overview of the key AI Analyst detections reported. Not only did it report on the SSL and HTTP C2 traffic, but it also reported on the payload downloads:

Cyber AI Analyst breaks down the APT41 attack
Figure 2: SSL C2 detection by Cyber AI Analyst
Figure 3: Payload detection by Cyber AI Analyst

Initial compromise

The initial compromise began with the successful exploitation of the Zoho ManageEngine zero-day vulnerability CVE-2020-10189. Following the initial intrusion, the Microsoft BITSAdmin command line tool was used to fetch and install a malicious Batch file, described below:

install.bat (MD5: 7966c2c546b71e800397a67f942858d0) from infrastructure 66.42.98[.]220 on port 12345.

Source: 10.60.50.XX
Destination: 66.42.98[.]220
Destination Port: 12345
Content Type: application/x-msdownload
Protocol: HTTP
Host: 66.42.98[.]220
URI: /test/install.bat
Method: GET
Status Code: 200

Figure 4: Outbound connection fetching batch file

Shortly after the initial compromise, the first stage Cobalt Strike Beacon LOADER was downloaded.

Cobalt Strike Beacon loader screenshot
Figure 5: Detection of the Cobalt Strike Beacon LOADER

Command and Control traffic

Interestingly, TeamViewer activity and the download of Notepad++ was taking place at the same time as the C2 traffic was starting in some of the customer attacks. This indicates APT41 trying to use familiar tools instead of completely ‘Living off the Land’.

Storesyncsvc.dll was a Cobalt Strike Beacon implant (trial-version) which connected to exchange.dumb1[.]com. A successful DNS resolution to 74.82.201[.]8 was identified, which Darktrace discerned as a successful SSL connection to a hostname with Dynamic DNS properties.

Multiple connections to exchange.dumb1[.]com were identified as beaconing to a C2 center. This C2 traffic to the initial Cobalt Strike Beacon was leveraged to download a second stage payload.

Interestingly, TeamViewer activity and the download of Notepad++ was taking place at the same time as the C2 traffic was starting in some of the customer attacks. This indicates APT41 trying to use familiar tools instead of completely ‘Living off the Land’. There is at least high certainty that the use of these two tools can be attributed to this intrusion instead of regular business activity. Notepad++ was not normally used in the target customers’ environments, nor was TeamViewer – in fact, the use of both applications was 100% unusual for the targeted organizations.

Attack tools download

CertUtil.exe, a command line program installed as part of Certificate Services, was then leveraged to connect externally and download the second stage payload.

Detection associated with Meterpreter activity

Figure 6: Darktrace detecting the usage of CertUtil

A few hours after this executable download, the infected device made an outbound HTTP connection requesting the URI /TzGG, which was identified as Meterpreter downloading further shellcode for the Cobalt Strike Beacon.

Figure 7: Detection associated with Meterpreter activity. No lateral movement or significant data exfiltration was observed.

How Cyber AI Analyst reported on the zero-day exploit

Darktrace not only detected this zero-day attack campaign, but Cyber AI Analyst also saved security teams valuable time by investigating disparate security events and generating a report that immediately put them in a position to take action.

The below screenshot shows the AI Analyst incidents reported in one infected environment, over the eight days covering the intrusion period. The first incident on the left represents the APT activity described here. The other five incidents are independent of the APT activity and not as severe.

AI Analyst Security Incidents
Figure 8: The security incidents surfaced by AI Analyst

AI Analyst reported on six incidents in total over the eight-day period. Each AI Analyst incident includes a detailed timeline and summary of the incident, in a concise format that takes an average of two minutes to review. This means that with Cyber AI Analyst, even a non-technical person could have actioned a response to this sophisticated, zero-day incident in less than five minutes.

Conclusion

Without public Indicators of Compromise (IoCs) or any open-source intelligence available, targeted attacks are incredibly difficult to detect. Moreover, even the best detections are useless if they cannot be actioned by a security analyst at an early stage. Too often this occurs because of an overwhelming volume of alerts, or simply because the skills barrier to triage and investigation is too high.

This appears to be a broad campaign to gain initial access to many different companies and sectors. While very sophisticated in nature, the threat sacrificed stealth for speed by targeting many companies at the same time. APT41 wanted to utilize the limited window of opportunity that the Zoho zero-day provided before IT staff starts patching.

Darktrace’s Cyber AI is specifically designed to detect the subtle signs of targeted, unknown attacks at an early stage, without relying on prior knowledge or IoCs. It achieves this by continuously learning the normal patterns of behavior for every user, device, and associated peer group from scratch, and ‘on the job’.

In the face of this zero-day attack campaign, the AI’s ability to (a) detect unknown threats with self-learning AI and (b) augment strained responders with AI-driven investigations and reporting proved crucial. Indeed, it ensured that the attacks were swiftly contained before escalating to the later stages of the attack lifecycle.

Indicators of Compromise

Selection of Darktrace model breaches:

  • Anomalous File / Script from Rare External
  • Anomalous File / EXE from Rare External Location
  • Compromise / SSL to DynDNS
  • Compliance / CertUtil External Connection
  • Anomalous Connection / CertUtil Requesting Non Certificate
  • Anomalous Connection / CertUtil to Rare Destination
  • Anomalous Connection / New User-Agent to IP Without Hostname
  • Device / Initial Breach Chain Compromise
  • Compromise / Slow Beaconing Activity To External Rare
  • Compromise / Beaconing Activity To External Rare
  • Anomalous File / Numeric Exe Download
  • Device / Large Number of Model Breaches
  • Anomalous Server Activity / Rare External from Server
  • Compromise / Sustained TCP Beaconing Activity To Rare Endpoint
  • Compliance / Remote Management Tool On Server

The below screenshot shows Darktrace model breaches occurring together during the compromise of one customer:

Figure 9: Darktrace model breaches occurring together

INSIDE THE SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
AUTHOR
ABOUT ThE AUTHOR
Max Heinemeyer
Chief Product Officer

Max is a cyber security expert with over a decade of experience in the field, specializing in a wide range of areas such as Penetration Testing, Red-Teaming, SIEM and SOC consulting and hunting Advanced Persistent Threat (APT) groups. At Darktrace, Max is closely involved with Darktrace’s strategic customers & prospects. He works with the R&D team at Darktrace, shaping research into new AI innovations and their various defensive and offensive applications. Max’s insights are regularly featured in international media outlets such as the BBC, Forbes and WIRED. Max holds an MSc from the University of Duisburg-Essen and a BSc from the Cooperative State University Stuttgart in International Business Information Systems.

Book a 1-1 meeting with one of our experts
share this article
USE CASES
No items found.
PRODUCT SPOTLIGHT
No items found.
COre coverage
No items found.

More in this series

No items found.

Blog

Inside the SOC

Following up on our Conversation: Detecting & Containing a LinkedIn Phishing Attack with Darktrace

Default blog imageDefault blog image
25
Jun 2024

Note: Real organization, domain and user names have been modified and replaced with fictitious names to maintain anonymity.  

Social media cyber-attacks

Social media is a known breeding ground for cyber criminals to easily connect with a near limitless number of people and leverage the wealth of personal information shared on these platforms to defraud the general public.  Analysis suggests even the most tech savvy ‘digital natives’ are vulnerable to impersonation scams over social media, as criminals weaponize brands and trends, using the promise of greater returns to induce sensitive information sharing or fraudulent payments [1].

LinkedIn phishing

As the usage of a particular social media platform increases, cyber criminals will find ways to exploit the increasing user base, and this trend has been observed with the rise in LinkedIn scams in recent years [2].  LinkedIn is the dominant professional networking site, with a forecasted 84.1million users by 2027 [3].  This platform is data-driven, so users are encouraged to share information publicly, including personal life updates, to boost visibility and increase job prospects [4] [5].  While this helps legitimate recruiters to gain a good understanding of the user, an attacker could also leverage the same personal content to increase the sophistication and success of their social engineering attempts.  

Darktrace detection of LinkedIn phishing

Darktrace detected a Software-as-a-Service (SaaS) compromise affecting a construction company, where the attack vector originated from LinkedIn (outside the monitoring of corporate security tools), but then pivoted to corporate email where a credential harvesting payload was delivered, providing the attacker with credentials to access a corporate file storage platform.  

Because LinkedIn accounts are typically linked to an individual’s personal email and are most commonly accessed via the mobile application [6] on personal devices that are not monitored by security teams, it can represent an effective initial access point for attackers looking to establish an initial relationship with their target. Moreover, user behaviors to ignore unsolicited emails from new or unknown contacts are less frequently carried over to platforms like LinkedIn, where interactions with ‘weak ties’ as opposed to ‘strong ties’ are a better predictor of job mobility [7]. Had this attack been allowed to continue, the threat actor could have leveraged access to further information from the compromised business cloud account to compromise other high value accounts, exfiltrate sensitive data, or defraud the organization.

LinkedIn phishing attack details

Reconnaissance

The initial reconnaissance and social engineering occurred on LinkedIn and was thus outside the purview of corporate security tools, Darktrace included.

However, the email domain “hausconstruction[.]com” used by the attacker in subsequent communications appears to be a spoofed domain impersonating a legitimate construction company “haus[.]com”, suggesting the attacker may have also impersonated an employee of this construction company on LinkedIn.  In addition to spoofing the domain, the attacker seemingly went further to register “hausconstruction.com” on a commercial web hosting platform.  This is a technique used frequently not just to increase apparent legitimacy, but also to bypass traditional security tools since newly registered domains will have no prior threat intelligence, making them more likely to evade signature and rules-based detections [8].  In this instance, open-source intelligence (OSINT) sources report that the domain was created several months earlier, suggesting this may have been part of a targeted attack on construction companies.  

Initial Intrusion

It was likely that during the correspondence over LinkedIn, the target user was solicited into following up over email regarding a prospective construction project, using their corporate email account.  In a probable attempt to establish a precedent of bi-directional correspondence so that subsequent malicious emails would not be flagged by traditional security tools, the attacker did not initially include suspicious links, attachments or use solicitous or inducive language within their initial emails.

Example of bi-directional email correspondence between the target and the attacker impersonating a legitimate employee of the construction company haus.com.
Figure 1: Example of bi-directional email correspondence between the target and the attacker impersonating a legitimate employee of the construction company haus.com.
Cyber AI Analyst investigation into one of the initial emails the target received from the attacker.
Figure 2: Cyber AI Analyst investigation into one of the initial emails the target received from the attacker.  

To accomplish the next stage of their attack, the attacker shared a link, hidden behind the inducing text “VIEW ALL FILES”, to a malicious file using the Hightail cloud storage service. This is also a common method employed by attackers to evade detection, as this method of file sharing does not involve attachments that can be scanned by traditional security tools, and legitimate cloud storage services are less likely to be blocked.

OSINT analysis on the malicious link link shows the file hosted on Hightail was a HTML file with the associated message “Following up on our LinkedIn conversation”.  Further analysis suggests the file contained obfuscated Javascript that, once opened, would automatically redirect the user to a malicious domain impersonating a legitimate Microsoft login page for credential harvesting purposes.  

The malicious HTML file containing obfuscated Javascript, where the highlighted string references the malicious credential harvesting domain.
Figure 3: The malicious HTML file containing obfuscated Javascript, where the highlighted string references the malicious credential harvesting domain.
Screenshot of fraudulent Microsoft Sign In page hosted on the malicous credential harvesting domain.
Figure 4: Screenshot of fraudulent Microsoft Sign In page hosted on the malicious credential harvesting domain.

Although there was prior email correspondence with the attacker, this email was not automatically deemed safe by Darktrace and was further analyzed for unusual properties and unusual communications for the recipient and the recipient’s peer group.  

Darktrace determined that:

  • It was unusual for this file storage solution to be referenced in communications to the user and the wider network
  • Textual properties of the email body suggested a high level of inducement from the sender, with a high level of focus on the phishing link.
  • The full link contained suspicious properties suggesting it is high risk.
Darktrace’s analysis of the phishing email, presenting key information about the unusual characteristics of this email, information on highlighted content, and an overview of actions that were initially applied.
Figure 5: Darktrace’s analysis of the phishing email, presenting key information about the unusual characteristics of this email, information on highlighted content, and an overview of actions that were initially applied.  

Based on these anomalies, Darktrace initially moved the phishing email to the junk folder and locked the link, preventing the user from directly accessing the malicious file hosted on Hightail.  However, the customer’s security team released the email, likely upon end-user request, allowing the target user to access the file and ultimately enter their credentials into that credential harvesting domain.

Darktrace alerts triggered by the malicious phishing email and the corresponding Autonomous Response actions.
Figure 6: Darktrace alerts triggered by the malicious phishing email and the corresponding Autonomous Response actions.

Lateral Movement

Correspondence between the attacker and target continued for two days after the credential harvesting payload was delivered.  Five days later, Darktrace detected an unusual login using multi-factor authentication (MFA) from a rare external IP and ASN that coincided with Darktrace/Email logs showing access to the credential harvesting link.

This attempt to bypass MFA, known as an Office365 Shell WCSS attack, was likely achieved by inducing the target to enter their credentials and legitimate MFA token into the fake Microsoft login page. This was then relayed to Microsoft by the attacker and used to obtain a legitimate session. The attacker then reused the legitimate token to log into Exchange Online from a different IP and registered the compromised device for MFA.

Screenshot within Darktrace/Email of the phishing email that was released by the security team, showing the recipient clicked the link to file storage where the malicious payload was stored.
Figure 7: Screenshot within Darktrace/Email of the phishing email that was released by the security team, showing the recipient clicked the link to file storage where the malicious payload was stored.
Event Log showing a malicious login and MFA bypass at 17:57:16, shortly after the link was clicked.  Highlighted in green is activity from the legitimate user prior to the malicious login, using Edge.
Figure 8: Event Log showing a malicious login and MFA bypass at 17:57:16, shortly after the link was clicked.  Highlighted in green is activity from the legitimate user prior to the malicious login, using Edge. Highlighted in orange and red is the malicious activity using Chrome.

The IP addresses used by the attacker appear to be part of anonymization infrastructure, but are not associated with any known indicators of compromise (IoCs) that signature-based detections would identify [9] [10].

In addition to  logins being observed within half an hour of each other from multiple geographically impossible locations (San Francisco and Phoenix), the unexpected usage of Chrome browser, compared to Edge browser previously used, provided Darktrace with further evidence that this activity was unlikely to originate from the legitimate user.  Although the user was a salesperson who frequently travelled for their role, Darktrace’s Self-Learning AI understood that the multiple logins from these locations was highly unusual at the user and group level, and coupled with the subsequent unexpected account modification, was a likely indicator of account compromise.  

Accomplish mission

Although the email had been manually released by the security team, allowing the attack to propagate, additional layers of defense were triggered as Darktrace's Autonomous Response initiated “Disable User” actions upon detection of the multiple unusual logins and the unauthorized registration of security information.  

However, the customer had configured Autonomous Response to require human confirmation, therefore no actions were taken until the security team manually approved them over two hours later. In that time, access to mail items and other SharePoint files from the unusual IP address was detected, suggesting a potential loss of confidentiality to business data.

Advanced Search query showing several FilePreviewed and MailItemsAccessed events from either the IPs used by the attacker, or using the software Chrome.  Note some of the activity originated from Microsoft IPs which may be whitelisted by traditional security tools.
Figure 9: Advanced Search query showing several FilePreviewed and MailItemsAccessed events from either the IPs used by the attacker, or using the software Chrome.  Note some of the activity originated from Microsoft IPs which may be whitelisted by traditional security tools.

However, it appears that the attacker was able to maintain access to the compromised account, as login and mail access events from 199.231.85[.]153 continued to be observed until the afternoon of the next day.  

Conclusion

This incident demonstrates the necessity of AI to security teams, with Darktrace’s ActiveAI Security Platform detecting a sophisticated phishing attack where human judgement fell short and initiated a real-time response when security teams could not physically respond as fast.  

Security teams are very familiar with social engineering and impersonation attempts, but these attacks remain highly prevalent due to the widespread adoption of technologies that enable these techniques to be deployed with great sophistication and ease.  In particular, the popularity of information-rich platforms like LinkedIn that are geared towards connecting with unknown people make it an attractive initial access point for malicious attackers.

In the second half of 2023 alone, over 200 thousand fake profiles were reported by members on LinkedIn [11].  Fake profiles can be highly sophisticated, use professional images, contain compelling descriptions, reference legitimate company listings and present believable credentials.  

It is unrealistic to expect end users to defend themselves against such sophisticated impersonation attempts. Moreover, it is extremely difficult for human defenders to recognize every fraudulent interaction amidst a sea of fake profiles. Instead, defenders should leverage AI, which can conduct autonomous investigations without human biases and limitations. AI-driven security can ensure successful detection of fraudulent or malicious activity by learning what real users and devices look like and identifying deviations from their learned behaviors that may indicate an emerging threat.

Appendices

Darktrace Model Detections

DETECT/ Apps

SaaS / Compromise / SaaS Anomaly Following Anomalous Login

SaaS / Compromise / Unusual Login and Account Update

SaaS / Unusual Activity / Multiple Unusual External Sources For SaaS Credential

SaaS / Access / Unusual External Source for SaaS Credential Use

SaaS / Compliance / M365 Security Information Modified

RESPOND/ Apps

Antigena / SaaS / Antigena Suspicious SaaS Activity Block

Antigena / SaaS / Antigena Unusual Activity Block

DETECT & RESPOND/ Email

·      Link / High Risk Link + Low Sender Association

·      Link / New Correspondent Classified Link

·      Link / Watched Link Type

·      Antigena Anomaly

·      Association / Unknown Sender

·      History / New Sender

·      Link / Link to File Storage

·      Link / Link to File Storage + Unknown Sender

·      Link / Low Link Association

List of IoCs

·      142.252.106[.]251 - IP            - Possible malicious IP used by attacker during cloud account compromise

·      199.231.85[.]153 – IP - Probable malicious IP used by attacker during cloud account compromise

·      vukoqo.hebakyon[.]com – Endpoint - Credential harvesting endpoint

MITRE ATT&CK Mapping

·      Resource Development - T1586 - Compromise Accounts

·      Resource Development - T1598.003 – Spearphishing Link

·      Persistence - T1078.004 - Cloud Accounts

·      Persistence - T1556.006 - Modify Authentication Process: Multi-Factor Authentication

·      Reconnaissance - T1593.001 – Social Media

·      Reconnaissance - T1598 – Phishing for Information

·      Reconnaissance - T1589.001 – Credentials

·      Reconnaissance - T1591.002 – Business Relationships

·      Collection - T1111 – Multifactor Authentication Interception

·      Collection - T1539 – Steal Web Session Cookie

·      Lateral Movement - T1021.007 – Cloud Services

·      Lateral Movement - T1213.002 - Sharepoint

References

[1] Jessica Barker, Hacked: The secrets behind cyber attacks, (London: Kogan Page, 2024), p. 130-146.

[2] https://www.bitdefender.co.uk/blog/hotforsecurity/5-linkedin-scams-and-how-to-avoid-them/

[3] https://www.washingtonpost.com/technology/2023/08/31/linkedin-personal-posts/

[4] https://www.forbes.com/sites/joshbersin/2012/05/21/facebook-vs-linkedin-whats-the-difference/

[5] https://thelinkedblog.com/2022/3-reasons-why-you-should-make-your-profile-public-1248/

[6] https://www.linkedin.com/pulse/50-linkedin-statistics-every-professional-should-ti9ue

[7] https://www.nytimes.com/2022/09/24/business/linkedin-social-experiments.html

[8] https://darktrace.com/blog/the-domain-game-how-email-attackers-are-buying-their-way-into-inboxes

[9] https://spur.us/context/142.252.106[.]251

[10] https://spur.us/context/199.231.85[.]153

[11]https://www.statista.com/statistics/1328849/linkedin-number-of-fake-accounts-detected-and-removed

Continue reading
About the author
Nicole Wong
Cyber Security Analyst

Blog

No items found.

Let the Dominos Fall! SOC and IR Metrics for ROI

Default blog imageDefault blog image
25
Jun 2024

One of the most enjoyable discussions (and debates) I engage in is the topic of Security Operations Center (SOC) and Incident Response (IR) metrics to measure and validate an organization’s Return on Investment (ROI). The debate part comes in when I hear vendor experts talking about “the only” SOC metrics that matter, and only list the two most well-known, while completely ignoring metrics that have a direct causal relationship.

In this blog, I will discuss what I believe are the SOC/IR metrics that matter, how each one has a direct impact on the others, and why organizations should ensure they are working towards the goal of why these metrics are measured in the first place: Reduction of Risk and Costs.

Reduction of Risk and Costs

Every security solution and process an organization puts in place should reduce the organization’s risk of a breach, exposure by an insider threat, or loss of productivity. How an organization realizes net benefits can be in several ways:

  • Improved efficiencies can result in SOC/IR staff focusing on other areas such as advanced threat hunting rather than churning through alerts on their security consoles. It may also help organizations dealing with the lack of skilled security staff by using Artificial Intelligence (AI) and automated processes.
  • A well-oiled SOC/IR team that has greatly reduced or even eliminated mundane tasks attracts, motivates, and retains talent resulting in reduced hiring and training costs.
  • The direct impact of a breach such as a ransomware attack can be devastating. According to the 2024 Data Breach Investigations Report by Verizon, MGM Resorts International reported the ALPHV ransomware cost the company approximately $100 million[1].
  • Failure to take appropriate steps to protect the organization can result in regulatory fines; and if an organization has, or is considering, purchasing Cyber Insurance, can result in declined coverage or increased premiums.

How does an organization demonstrate they are taking proactive measures to prevent breaches? That is where it's important to understand the nine (yes, nine) key metrics, and how each one directly influences the others, play their roles.

Metrics in the Incident Response Timeline

Let’s start with a review of the key steps in the Incident Response Timeline:

Seven of the nine key metrics are in the IR timeline, while two of the metrics occur before you ever have an incident. They occur in the Pre-Detection Stage.

Pre-Detection stage metrics are:

  • Preventions Per Intrusion Attempt (PPIA)
  • False Positive Reduction Rate (FPRR)

Next is the Detect and Investigate stage, there are three metrics to consider:

  • Mean Time to Detection (MTTD)
  • Mean Time to Triage (MTTT)
  • Mean Time to Understanding (MTTU)

This is followed by the Remediation stage, there are two metrics here:

  • Mean Time to Containment (MTTC)
  • Mean Time to Remediation / Recovery (MTTR)

Finally, there is the Risk Reduction stage, there are two metrics:

  • Mean Time to Advice (MTTA)
  • Mean Time to Implementation (MTTI)

Pre-Detection Stage

Preventions Per Intrusion Attempt

PPIA is defined as stopping any intrusion attempt at the earliest possible stage. Your network Intrusion Prevention System (IPS) blocks vulnerability exploits, your e-mail security solution intercepts and removes messages with malicious attachments or links, your egress firewall blocks unauthorized login attempts, etc. The adversary doesn’t get beyond Step 1 in the attack life cycle.

This metric is the first domino. Every organization should strive to improve on this metric every day. Why? For every intrusion attempt you stop right out of the gate, you eliminate the actions for every other metric. There is no incident to detect, triage, investigate, remediate, or analyze post-incident for ways to improve your security posture.

When I think about PPIA, I always remember back to a discussion with a former mentor, Tim Crothers, who discussed the benefits of focusing on Prevention Failure Detection.

The concept is that as you layer your security defenses, your PPIA moves ever closer to 100% (no one has ever reached 100%). This narrows the field of fire for adversaries to breach into your organization. This is where novel, unknown, and permuted threats live and breathe. This is where solutions utilizing Unsupervised Machine Learning excel in raising anomalous alerts – indications of potential compromise involving one of these threats. Unsupervised ML also raises alerts on anomalous activity generated by known threats and can raise detections before many signature-based solutions. Most organizations struggle to find strong permutations of known threats, insider threats, supply chain attacks, attacks utilizing n-day and 0-day exploits. Moving PPIA ever closer to 100% also frees your team up for conducting threat hunting activities – utilizing components of your SOC that collect and store telemetry to query for potential compromises based on hypothesis the team raises. It also significantly reduces the alerts your team must triage and investigate – solving many of the issues outlined at the start of this paper.

False Positive Reduction Rate

Before we discuss FPRR, I should clarify how I define False Positives (FPs). Many define FPs as an alert that is in error (i.e.: your EDR alerts on malware that turns out to be AV signature files). While that is a FP, I extend the definition to include any alert that did not require triage / investigation and distracts the SOC/IR team (meaning they conducted some level of triage / investigation).

This metric is the second domino. Why is this metric important? Every alert your team exerts time and effort on that is a non-issue distracts them from alerts that matter. One of the major issues that has resonated in the security industry for decades is that SOCs are inundated with alerts and cannot clear the backlog. When it comes to PPIA + FPRR, I have seen analysts spend time investigating alerts that were blocked out of the gate while their screen continued to fill up with more. You must focus on Prevention Failure Detection to get ahead of the backlog.

Detect and Investigate Stages

Mean Time to Detection

MTTD, or “Dwell Time”, has decreased dramatically over the past 12 years. From well over a year to 16 days in 2023[2]. MTTD is measured from the earliest possible point you could detect the intrusion to the moment you actually detect it.

This third domino is important because the longer an adversary remains undetected, the more the odds increase they will complete their mission objective. It also makes the tasks of triage and investigation more difficult as analysts must piece together more activity and adversaries may be erasing evidence along the way – or your storage retention does not cover the breach timeline.

Many solutions focusing solely on MTTD can actually create the very problem SOCs are looking to solve.  That is, they generate so much alerting that they flood the console, email, or text messaging app causing an unmanageable queue of alerts (this is the problem XDR solutions were designed to resolve by focusing on incidents rather than alerts).

Mean Time to Triage

MTTT involves SOCs that utilize Level 1 (aka Triage) analysts to render an “escalate / do not escalate” alert verdict accurately. Accuracy is important because Triage Analysts typically are staff new to cyber security (recent grad / certification) and may over escalate (afraid to miss something important) or under escalate (not recognize signs of a successful breach). Because of this, a small MTTT does not always equate to successful handling of incidents.

This metric is important because keeping your senior staff focused on progressing incidents in a timely manner (and not expending time on false positives) should reduce stress and required headcount.

Mean Time to Understanding

MTTU deals with understanding the complete nature of the incident being investigated. This is different than MTTT which only deals with whether the issue merits escalation to senior analysts. It is then up to the senior analysts to determine the scope of the incident, and if you are a follower of my UPSET Investigation Framework, you know understanding the full scope involves:

U = All compromised accounts

P = Persistence Mechanisms used

S = All systems involved (organization, adversary, and intermediaries)

E = Endgame (or mission objective)

T = Techniques, Tactics, Procedures (TTPs) utilized by the adversary

MTTU is important because this information is critical before any containment or remediation actions are taken. Leave a stone unturned, and you alert the adversary that you are onto them and possibly fail to close an avenue of access.

Remediation Stages

Mean Time to Containment

MTTC deals with neutralizing the threat. You may not have kicked the adversary out, but you have halted their progress to their mission objective and ability to inflict further damage. This may be through use of isolation capabilities, termination of malicious processes, or firewall blocks.

MTTC is important, especially with ransomware attacks where every second counts. Faster containment responses can result in reduced / eliminated disruption to business operations or loss of data.

Mean Time to Remediation / Recovery

The full scope of the incident is understood, the adversary has been halted in their tracks, no malicious processes are running on any systems in your organization. Now is the time to put things back to right. MTTR deals with the time involved in restoring business operations to pre-incident stage. It means all remnants of changes made by the adversary (persistence, account alterations, programs installed, etc.) are removed; all disrupted systems are restored to operations (i.e.: ransomware encrypted systems are recovered from backups / snapshots), compromised user accounts are reset, etc.

MTTR is important because it informs senior management of how fast the organization can recover from an incident. Disaster Recovery and Business Continuity plans play a major role in improving this score.

Risk Reduction Stages

Mean Time to Advice

After the dust has settled from the incident, the job is not done. MTTA deals with identifying and assessing the specific areas (vulnerabilities, misconfigurations, lack of security controls) that permitted the adversary to advance to the point where detection occurred (and any actions beyond). The SOC and IR teams should then compile a list of recommendations to present to management to improve the security posture of the organization so the same attack path cannot be used.

Mean Time to Implement

Once recommendations are delivered to management, how long does it take to implement them? MTTI tracks this timeline because none of it matters if you don’t fix the holes that led to the breach.

Nine Dominos

There are the nine dominos of SOC / IR metrics I recommend helping organizations know if they are on the right track to reduce risk, costs and improve morale / retention of the security teams. You may not wish to track all nine, but understanding how each metric impacts the others can provide visibility into why you are not seeing expected improvements when you implement a new security solution or change processes.

Improving prevention and reducing false positives can make huge positive impacts on your incident response timeline. Utilizing solutions that get you to resolution quicker allows the team to focus on recommendations and risk reduction strategies.

Whichever metrics you choose to track, just be sure the dominos fall in your favor.

References

[1] 2024 Verizon Data Breach Investigations Report, p83

[2] Mandiant M-Trends 2023

Continue reading
About the author
John Bradshaw
Sr. Director, Technical Marketing
Our ai. Your data.

Elevate your cyber defenses with Darktrace AI

Start your free trial
Darktrace AI protecting a business from cyber threats.